Video prediction encoding device, video prediction encoding method, video prediction encoding program, video prediction decoding device, video prediction decoding method, and video prediction decoding program

ABSTRACT

A predicted signal generation unit provided in a video predictive encoding device estimates a zero-th motion vector for derivation of a zero-th predicted signal, selects a zero-th motion vector predictor similar to the zero-th motion vector, and generates zero-th side information containing a zero-th motion vector predictor index to identify the motion vector predictor and a motion vector difference determined from the zero-th motion vector and the zero-th motion vector predictor. The video predictive encoding device selects a motion vector for generation of a first predicted signal having a high correlation with a target region, generates first side information containing a first motion vector predictor index to identify the motion vector as a first motion vector predictor, sets the first motion vector predictor to a first motion vector, and combines the zero-th and first predicted signals to generate a predicted signal of the target region.

This application is a continuation of U.S. application Ser. No.15/473,791, filed Mar. 30, 2017, which is a continuation of U.S.application Ser. No. 15/357,696, filed Nov. 21, 2016, which is acontinuation of U.S. application Ser. No. 14/271,423, filed May 6, 2014,which is a continuation of PCT/JP2012/074575, filed Sep. 25, 2012, whichclaims the benefit of the filing date pursuant to 35 U.S.C. § 119(e) ofJP2011-243490, filed Nov. 7, 2011, all of which are incorporated hereinby reference.

TECHNICAL FIELD

The present disclosure relates to a video predictive encoding device, avideo predictive encoding method, a video predictive encoding program, avideo predictive decoding device, a video predictive decoding method,and a video predictive decoding program and, more particularly, thepresent disclosure relates to a video predictive encoding device, avideo predictive encoding method, a video predictive encoding program, avideo predictive decoding device, a video predictive decoding method,and a video predictive decoding program to generate a final blockpredicted signal (bi-predicted signal) by averaging two predictedsignals.

BACKGROUND ART

Compression encoding technologies are used for efficient transmissionand storage of still pictures and video data. The techniques defined inMPEG-1 to 4 and ITU (International Telecommunication Union) H.261 toH.264 are commonly used for video data.

SUMMARY

Using encoding techniques, a picture which is used as an encoding targetis divided into a plurality of blocks and then an encoding process and adecoding process are carried out on a block basis. In intra-framepredictive encoding, a predicted signal is generated using apreviously-reproduced neighboring picture signal (a signal reconstructedfrom compressed picture data) present in the same frame as a targetblock and then a residual signal obtained by subtracting the predictedsignal from a signal of the target block is encoded. In inter-framepredictive encoding, a predicted signal is generated with compensationfor motion with reference to a previously-reproduced neighboring picturesignal present in a frame different from a target block, and a residualsignal obtained by subtracting the predicted signal from a signal of thetarget block is encoded.

Intra-frame predictive encoding, such as, for example, in H.264, employsa method of generating the predicted signal by extrapolatingpreviously-reproduced pixel values adjacent to a block as an encodingtarget, in predetermined directions. FIG. 20 is a schematic diagram forexplaining an example intra-frame prediction method such as the methodused in ITU H.264. In FIG. 20(A), a target block 802 is a block as anencoding target, and a pixel group 801 consisting of pixels A to Madjacent to a boundary of the target block 802 is a neighboring region,which is a picture signal previously reproduced in past processing.

In this case, the predicted signal is generated by downwardly extendingthe pixel group 801 as neighboring pixels located immediately above thetarget block 802. In FIG. 20(B), the predicted signal is generated byrightwardly extending previously-reproduced pixels (I to L) located onthe left side of the target block 804. Methods for generation of thepredicted signal are described, for example, in U.S. Pat. No. 6,765,964.A difference is calculated between each of nine predicted signalsgenerated by the methods shown in FIG. 20(A) to (I) in theabove-described manner, and the pixel signal of the target block, andone with the minimum difference is selected as an optimum predictedsignal. As described above, the predicted signal can be generated byextrapolation of pixels.

In inter-frame predictive encoding, the predicted signal can begenerated by a method of searching previously-reproduced frames for asignal similar to the pixel signal of the block as an encoding target.Then a motion vector is encoded as a spatial displacement amount betweenthe target block and a region composed of the detected signal, and aresidual signal between the pixel signal of the target block and thepredicted signal. The searching technique for the motion vector for eachblock as described above is called block matching.

FIG. 19 is a schematic diagram for explaining an example of a blockmatching process. The below will describe a procedure for generating apredicted signal for an example of a target block 702 on an encodingtarget frame 701. A reference frame 703 is a previously-reproducedpicture and a region 704 is a region located spatially at the sameposition as the target block 702. In the block matching, a search range705 including the region 704 is set and a region 706 with the minimumsum of absolute errors from the pixel signal of the target block 702 isdetected from pixel signals in this search range 705. The signal of thisregion 706 is determined to be a predicted signal, and a displacementamount from the region 704 to the region 706 is detected as a motionvector 707. Also a method of preparing a plurality of reference frames703, selecting a reference frame to be applied to block matching foreach target block, and detecting reference frame selection informationcan be employed. In H.264, for example, a plurality of prediction typesof different block sizes for encoding of motion vectors can be prepared,in order to adapt for local characteristic changes of pictures. Theprediction types of H.264 are described, for example, in U.S. Pat. No.7,003,035.

In compression encoding of video data, an encoding order of pictures(frames or fields) may be optional. For this reason, there are twotechniques regarding the encoding order in inter-frame prediction whichcan be used to generate the predicted signal with reference topreviously-reproduced frames. The first technique is uni-prediction,which generates the predicted signal with reference to onepreviously-reproduced frame, and the second technique is bi-prediction,which averages two predicted signals obtained with reference to one ortwo previously-reproduced frames. The uni-prediction technique includesforward prediction with reference to a past previously-reproduced framein a display order, and backward prediction with reference to a futurepreviously-reproduced frame in the display order. Examples of thesetypes of inter-frame predictions are described, for example, in U.S.Pat. No. 6,259,739.

In an example, such as in H.264, the second inter-frame technique(bi-prediction) is performed by creating two reference frame lists eachconsisting of a plurality of previously-reproduced frames as candidatesfor reference picture 703. Block matching is carried out with theplurality of reference frames registered in each reference picture listto detect two regions corresponding to the region 706, and two predictedsignals thus detected are averaged.

Examples of reference picture lists will be described with reference toFIGS. 5 and 6. In FIG. 5(A), a frame 505 indicates an encoding targetpicture and frames 501 to 504 indicate previously-reproduced frames. InFIG. 5(B), a frame 510 represents an encoding target frame and frames507, 508, 509, and 511 represent previously-reproduced frames. Eachpicture (frame) is identified by a frame number (frame_num). In FIG. 6List0 and List1 indicate two reference frame lists, FIG. 6(A) shows anexample of reference frame lists for FIG. 5(A), and FIGS. 6(B) and (C)show examples of reference frame lists for FIG. 5(B). In FIGS. 6(A) and(C), four reference frames are registered in each reference frame listand in FIG. 6(B) two reference frames are registered in each referenceframe list. Each reference frame is identified by a reference frameindex (ref_idx). Any previously-reproduced pictures can be registered inthe reference picture lists. In the present specification, as anon-limiting example in order to facilitate an understanding of thecontents, notations of zero-th motion information and first motioninformation are used according to the foregoing reference frame listsList0 and List1.

In bi-prediction, noise in the predicted signal can be removed by asmoothing effect based on averaging of two similar predicted signals.However, blocks which the smoothing effect benefits most are oftentexture regions or flat regions including noise, and reference framescontain signals similar to those in the blocks.

Since signals of these regions have strong randomness, motion vectorsbetween neighboring blocks can vary significantly when two predictedsignals similar to a target block in these regions are detected from aplurality of reference frames. Since a motion vector is encoded as adifference from a motion vector of a neighboring block, the variation inmotion vectors between neighboring blocks can lead to an increase in anamount of coding bits thereof.

As an example technique to reduce the amount of coding bits in thebi-prediction, there is a method of deriving two motion vectorsaccording to conditions of neighboring blocks on the decoding side. Itis, however, difficult to enhance the similarity of two predictedsignals because of strong restrictions on available predicted signals,thus failing to achieve the satisfactory smoothing effect.

A video predictive coding system that includes a video predictiveencoding device, a video predictive encoding method, a video predictiveencoding program, a video predictive decoding device, a video predictivedecoding method, and a video predictive decoding program capable ofefficiently suppressing the noise of the predicted signal with a smalleramount of coding bits for bi-prediction to encode two motion vectors, byencoding one motion vector to generate the predicted signal similar tothe target block and selectively determining the other motion vectorfrom previously-encoded motion information.

An example embodiment of the video predictive coding system includes avideo predictive encoding device comprising: region division means whichdivides an input picture into a plurality of regions; predicted signalgeneration means which determines a motion vector for deriving from apreviously-reproduced picture a signal having a high correlation with atarget region as an encoding target, out of the regions divided by theregion division means; motion information storing means which stores themotion vector; residual signal generation means which generates aresidual signal between a predicted signal of the target region and apixel signal of the target region; residual signal compression meanswhich compresses the residual signal generated by the residual signalgeneration means; residual signal reconstruction means which generates areproduced residual signal by reconstruction from compressed data of theresidual signal; encoding means which encodes a motion vector predictorsimilar to a motion vector of the target region selected frompreviously-reproduced motion vectors stored in the motion informationstoring means, side information determined from the motion vector of thetarget region, and the compressed data of the residual signal; andpicture storing means which adds the predicted signal to the reproducedresidual signal to reconstruct a pixel signal of the target region, andwhich stores the reconstructed pixel signal as the previously-reproducedpicture, wherein the motion vector includes a zero-th motion vector usedfor generation of a zero-th predicted signal, and a first motion vectorused for generation of a first predicted signal, and wherein thepredicted signal generation means comprises: zero-th motion informationestimation means which estimates the zero-th motion vector forderivation of the zero-th predicted signal, which selects a zero-thmotion vector predictor similar to the estimated zero-th motion vectorfrom a plurality of motion vectors stored in the motion informationstoring means, and which generates zero-th side information containing azero-th motion vector predictor index to identify the motion vectorpredictor selected, and a motion vector difference determined from thezero-th motion vector and the zero-th motion vector predictor; firstmotion information estimation means which selects a motion vector forgeneration of the first predicted signal having a high correlation withthe target region, from a plurality of motion vectors stored in themotion information storing means, which generates first side informationcontaining a first motion vector predictor index to identify theselected motion vector as a first motion vector predictor, and whichsets the first motion vector predictor to the first motion vector; andpredicted signal combining means which combines the zero-th predictedsignal and the first predicted signal to generate the predicted signalof the target region.

The above video predictive encoding device may be configured in anembodiment wherein the first motion information estimation means furtherincludes functions to estimate a second motion vector for derivation ofthe first predicted signal, to thereby detect the second motion vector,to select a second motion vector predictor similar to the estimatedsecond motion vector, from a plurality of motion vectors stored in themotion information storing means, and to generate second sideinformation containing a second motion vector predictor index toidentify the selected motion vector predictor, and a motion vectordifference determined from the second motion vector and the secondmotion vector predictor, wherein the predicted signal generation meansgenerates the first predicted signal using the first motion vector whena plurality of previously-reproduced picture stored in the picturestoring means all are past pictures in a display order with respect toan encoding target picture, and the predicted signal generation meansgenerates the first predicted signal using the second motion vector whena plurality of previously-reproduced pictures stored in the picturestoring means include a future picture in the display order with respectto the encoding target picture, and wherein the encoding means encodesindication information to indicate that the first side information isencoded, in each frame or in each slice when a plurality ofpreviously-reproduced pictures stored in the picture storing means allare past pictures in the display order with respect to the encodingtarget picture, and the encoding means encodes indication information toindicate that the second side information is encoded, in each frame orin each slice when a plurality of previously-reproduced pictures storedin the picture storing means include a future picture in the displayorder with respect to the encoding target picture; and wherein theencoding means encodes as side information of each target region, thezero-th side information, and either the first side information or thesecond side information based on the indication information.

An example embodiment of the video predictive coding system includes avideo predictive decoding device comprising: decoding means whichdecodes a compressed data out of plural sets of compressed data obtainedby encoding a plurality of divided regions, the compressed dataincluding side information and a residual signal of a target region,which is a target to be decoded; motion information reconstruction meanswhich reconstructs a motion vector used to generate a predicted signalof the target region from the side information; motion informationstoring means which stores the motion vector; motion compensation meanswhich generates the predicted signal of the target region, based on themotion vector; residual signal reconstruction means which reconstructs areproduced residual signal of the target region from the compressed dataof the residual signal; and picture storing means which adds thepredicted signal to the reproduced residual signal to reconstruct apixel signal of the decoding target region and which stores thereconstructed pixel signal as a previously-reproduced picture, whereinthe decoding means decodes zero-th side information and first sideinformation, wherein the zero-th side information contains a zero-thmotion vector difference, and a zero-th motion vector predictor index toidentify as a zero-th motion vector predictor one motion vector selectedfrom a plurality of motion vectors stored in the motion informationstoring means, wherein the first side information contains a firstmotion vector predictor index to identify as a first motion vectorpredictor one motion vector selected from a plurality of motion vectorsstored in the motion information storing means, wherein the motioninformation reconstruction means comprises: zero-th motion informationreconstruction means which generates the zero-th motion vectorpredictor, based on the zero-th motion vector predictor index, and whichadds the generated zero-th motion vector predictor to the zero-th motionvector difference to reconstruct a zero-th motion vector; and firstmotion information reconstruction means which generates the first motionvector predictor, based on the first motion vector predictor index, toreconstruct the generated first motion vector predictor as a firstmotion vector, and wherein the motion compensation means combines twosignals obtained from the previously-reproduced picture, based on thezero-th motion vector and the first motion vector, to generate thepredicted signal of the target region.

The above video predictive decoding device may be configured in anembodiment wherein the decoding means further decodes indicationinformation to indicate whether the first side information contains amotion vector difference, in each frame or in each slice, wherein whenthe indication information indicates that the first side informationcontains a first motion vector difference, the decoding means decodesthe motion vector difference as the first side information, and wherein,when the indication information indicates that the first sideinformation does not contain the first motion vector difference, thefirst motion information reconstruction means generates the first motionvector predictor, based on the first motion vector predictor index, andreconstructs the generated first motion vector predictor as the firstmotion vector; and wherein, when the indication information indicatesthat the first side information contains the vector difference, thefirst motion information reconstruction means generates the first motionvector predictor, based on the first motion vector predictor index, andadds the generated first motion vector predictor to the decoded motionvector difference to generate and reconstruct the first motion vector.

The video predictive coding system can also be understood as a videopredictive encoding method, a video predictive decoding method, a videopredictive encoding program, and a video predictive decoding program,which can be described as below.

An example embodiment of the video predictive encoding method can beexecuted by a video predictive encoding device, comprising: a regiondivision step of dividing an input picture into a plurality of regions;a predicted signal generation step of determining a motion vector forderiving from a previously-reproduced picture a signal having a highcorrelation with a target region as an encoding target, out of theregions divided by the region division step; a motion informationstoring step of storing the motion vector in motion information storingmeans; a residual signal generation step of generating a residual signalbetween a predicted signal of the target region and a pixel signal ofthe target region; a residual signal compression step of compressing theresidual signal generated by the residual signal generation step; aresidual signal reconstruction step of generating a reproduced residualsignal by reconstruction from compressed data of the residual signal; anencoding step of selecting and encoding a motion vector predictorsimilar to a motion vector of the target region, the target motionvector selected from: previously-reproduced motion vectors stored in themotion information storing means, side information determined from themotion vector of the target region, and the compressed data of theresidual signal; and a picture storing step of adding the predictedsignal to the reproduced residual signal to reconstruct a pixel signalof the target region, and storing the reconstructed pixel signal as thepreviously-reproduced picture in picture storing means, wherein themotion vector includes a zero-th motion vector used to generate azero-th predicted signal, and a first motion vector used to generate afirst predicted signal, and wherein the predicted signal generation stepcomprises: a zero-th motion information estimation step of estimatingthe zero-th motion vector for derivation of the zero-th predictedsignal, selecting a zero-th motion vector predictor similar to theestimated zero-th motion vector from a plurality of motion vectorsstored in the motion information storing step, and generating zero-thside information containing a zero-th motion vector predictor index toidentify the motion vector predictor selected, and a motion vectordifference determined from the zero-th motion vector and the zero-thmotion vector predictor; a first motion information estimation step ofselecting, from a plurality of motion vectors stored in the motioninformation storing step, a motion vector having a high correlation withthe target region for generation of the first predicted signal,generating first side information containing a first motion vectorpredictor index to identify the selected motion vector as a first motionvector predictor, and setting the first motion vector predictor as thefirst motion vector; and a predicted signal combining step of combiningthe zero-th predicted signal and the first predicted signal to generatethe predicted signal of the target region.

The above video predictive encoding method may be configured in anembodiment wherein in the first motion information estimation step, thevideo predictive encoding device further estimates a second motionvector for derivation of the first predicted signal, to detect thesecond motion vector, selects a second motion vector predictor similarto the estimated second motion vector from a plurality of motion vectorsstored in the motion information storing step, and generates second sideinformation containing a second motion vector predictor index toidentify the motion vector predictor selected, and a motion vectordifference determined from the second motion vector and the secondmotion vector predictor, wherein in the predicted signal generationstep, the video predictive encoding device generates the first predictedsignal using the first motion vector when a plurality ofpreviously-reproduced pictures stored in the picture storing means allare past pictures in a display order with respect to an encoding targetpicture, and the video predictive encoding device generates the firstpredicted signal using the second motion vector when a plurality ofpreviously-reproduced pictures stored in the picture storing meansinclude a future picture in the display order with respect to theencoding target picture, and wherein in the encoding step, when aplurality of previously-reproduced pictures stored in the picturestoring means all are past pictures in the display order with respect tothe encoding target picture, the video predictive encoding deviceencodes indication information to indicate that the first sideinformation is encoded, in each frame or in each slice; when a pluralityof previously-reproduced pictures stored in the picture storing meansinclude a future picture in the display order with respect to theencoding target picture, the video predictive encoding device encodesindication information to indicate that the second side information isencoded, in each frame or in each slice; and wherein in the encodingstep, the video predictive encoding device encodes as side informationof each target region, the zero-th side information, and either thefirst side information or the second side information based on theindication information.

An embodiment of the video predictive coding system includes a videopredictive decoding method executed by a video predictive decodingdevice, comprising: a decoding step of decoding a compressed data out ofplural sets of compressed data obtained by decoding a plurality ofdivided regions, the compressed data comprising side information and aresidual signal of a decoding target region which is target to bedecoded; a motion information reconstruction step of reconstructing amotion vector used to generate a predicted signal of the target regionfrom the side information; a motion information storing step of storingthe motion vector in motion information storing means; a motioncompensation step of generating the predicted signal of the targetregion, based on the motion vector; a residual signal reconstructionstep of reconstructing a reproduced residual signal of the target regionfrom the compressed data of the residual signal; and a picture storingstep of adding the predicted signal to the reproduced residual signal toreconstruct a pixel signal of the decoding target region, and storingthe reconstructed pixel signal as a previously-reproduced picture,wherein in the decoding step, the video predictive decoding devicedecodes zero-th side information and first side information, wherein thezero-th side information contains a zero-th motion vector difference,and a zero-th motion vector predictor index to identify as a zero-thmotion vector predictor one motion vector selected from a plurality ofmotion vectors stored in the motion information storing step, whereinthe first side information contains a first motion vector predictorindex to identify as a first motion vector predictor one motion vectorselected from a plurality of motion vectors stored in the motioninformation storing step, wherein the motion information reconstructionstep comprises: a zero-th motion information reconstruction step ofgenerating the zero-th motion vector predictor, based on the zero-thmotion vector predictor index, and adding the generated zero-th motionvector predictor to the zero-th motion vector difference to reconstructa zero-th motion vector; and a first motion information reconstructionstep of generating the first motion vector predictor, based on the firstmotion vector predictor index, to reconstruct the generated first motionvector predictor as a first motion vector, and wherein in the motioncompensation step, the video predictive decoding device combines twosignals derived from the previously-reproduced picture, based on thezero-th motion vector and the first motion vector, to generate thepredicted signal of the target region.

The above video predictive decoding method may be configured in anembodiment wherein in the decoding step, the video predictive decodingdevice further decodes indication information to indicate whether thefirst side information contains a motion vector difference, in eachframe or in each slice, wherein when the indication informationindicates that the first side information contains a first motion vectordifference, the video predictive decoding device decodes the motionvector difference as the first side information, and wherein in thefirst motion information reconstruction step, when the indicationinformation indicates that the first side information does not containthe first motion vector difference, the video predictive decoding devicegenerates the first motion vector predictor, based on the first motionvector predictor index, and reconstructs the generated first motionvector predictor as the first motion vector; and wherein in the firstmotion information reconstruction step, when the indication informationindicates that the first side information contains the first motionvector difference, the video predictive decoding device generates thefirst motion vector predictor, based on the first motion vectorpredictor index, and adds the generated first motion vector predictor tothe decoded motion vector difference to generate and reconstruct thefirst motion vector.

An embodiment of the video predictive coding system includes a videopredictive encoding program for letting a computer function as: regiondivision means which divides an input picture into a plurality ofregions; predicted signal generation means which determines a motionvector for deriving from a previously-reproduced picture a signal havinga high correlation with a target region as an encoding target, out ofthe regions divided by the region division means; motion informationstoring means which stores the motion vector; residual signal generationmeans which generates a residual signal between a predicted signal ofthe target region and a pixel signal of the target region; residualsignal compression means which compresses the residual signal generatedby the residual signal generation means; residual signal reconstructionmeans which generates a reproduced residual signal by reconstructionfrom compressed data of the residual signal; encoding means whichencodes a motion vector predictor, similar to a motion vector of thetarget region, selected from: previously-reproduced motion vectorsstored in the motion information storing means, side informationdetermined from the motion vector of the target region, and thecompressed data of the residual signal; and picture storing means whichadds the predicted signal to the reproduced residual signal toreconstruct a pixel signal of the target region, and which stores thereconstructed pixel signal as the previously-reproduced picture, whereinthe motion vector includes a zero-th motion vector used to generate azero-th predicted signal, and a first motion vector used to generate afirst predicted signal, and wherein the predicted signal generationmeans comprises: zero-th motion information estimation means whichestimates the zero-th motion vector for derivation of the zero-thpredicted signal, which selects a zero-th motion vector predictorsimilar to the estimated zero-th motion vector from a plurality ofmotion vectors stored in the motion information storing means, and whichgenerates zero-th side information containing a zero-th motion vectorpredictor index to identify the motion vector predictor selected, and amotion vector difference determined from the zero-th motion vector andthe zero-th motion vector predictor; first motion information estimationmeans which selects a motion vector for generation of the firstpredicted signal having a high correlation with the target region, froma plurality of motion vectors stored in the motion information storingmeans, which generates first side information containing a first motionvector predictor index to identify the selected motion vector as a firstmotion vector predictor, and which sets the first motion vectorpredictor to the first motion vector; and predicted signal combiningmeans which combines the zero-th predicted signal and the firstpredicted signal to generate the predicted signal of the target region.

The above video predictive encoding program may be configured in anembodiment wherein the first motion information estimation means furtherhas a function to estimate a second motion vector for derivation of thefirst predicted signal, to detect the second motion vector, to select asecond motion vector predictor similar to the second motion vectorestimated, from a plurality of motion vectors stored in the motioninformation storing means, and to generate second side informationcontaining a second motion vector predictor index to identify the motionvector predictor selected, and a motion vector difference determinedfrom the second motion vector and the second motion vector predictor,wherein the predicted signal generation means generates the firstpredicted signal using the first motion vector when a plurality ofpreviously-reproduced picture stored in the picture storing means allare past pictures in a display order with respect to an encoding targetpicture, and the predicted signal generation means generates the firstpredicted signal using the second motion vector when a plurality ofpreviously-reproduced pictures stored in the picture storing meansinclude a future picture in the display order with respect to theencoding target picture, and wherein the encoding means encodesindication information to indicate that the first side information isencoded, in each frame or in each slice when a plurality ofpreviously-reproduced pictures stored in the picture storing means allare past pictures in the display order with respect to the encodingtarget picture, and the encoding means encodes indication information toindicate that the second side information is encoded, in each frame orin each slice when a plurality of previously-reproduced pictures storedin the picture storing means include a future picture in the displayorder with respect to the encoding target picture; and wherein theencoding means encodes as side information of each target region, thezero-th side information, and either the first side information or thesecond side information based on the indication information.

An embodiment of the video predictive coding system includes a videopredictive decoding program for letting a computer function as: decodingmeans which decodes a compressed data out of plural sets of compresseddata obtained by decoding a plurality of divided regions, the compresseddata comprising side information and a residual signal of a decodingtarget region, which is a target to be decoded; motion informationreconstruction means which reconstructs a motion vector used to generatea predicted signal of the target region from the side information;motion information storing means which stores the motion vector; motioncompensation means which generates the predicted signal of the targetregion, based on the motion vector; residual signal reconstruction meanswhich reconstructs a reproduced residual signal of the target regionfrom the compressed data of the residual signal; and picture storingmeans which adds the predicted signal to the reproduced residual signalto reconstruct a pixel signal of the decoding target region and whichstores the reconstructed pixel signal as a previously-reproducedpicture, wherein the decoding means decodes zero-th side information andfirst side information, wherein the zero-th side information contains azero-th motion vector difference, and a zero-th motion vector predictorindex to identify as a zero-th motion vector predictor one motion vectorselected from a plurality of motion vectors stored in the motioninformation storing means, wherein the first side information contains afirst motion vector predictor index to identify as a first motion vectorpredictor one motion vector selected from a plurality of motion vectorsstored in the motion information storing means, wherein the motioninformation reconstruction means comprises: zero-th motion informationreconstruction means which generates the zero-th motion vectorpredictor, based on the zero-th motion vector predictor index, and whichadds the generated zero-th motion vector predictor to the zero-th motionvector difference to reconstruct a zero-th motion vector; and firstmotion information reconstruction means which generates the first motionvector predictor, based on the first motion vector predictor index, toreconstruct the generated first motion vector predictor as a firstmotion vector, and wherein the motion compensation means combines twosignals derived from the previously-reproduced picture, based on thezero-th motion vector and the first motion vector, to generate thepredicted signal of the target region.

The above video predictive decoding program may be configured in anembodiment wherein the decoding means further decodes indicationinformation to indicate whether the first side information contains amotion vector difference, in each frame or in each slice, wherein whenthe indication information indicates that the first side informationcontains a first motion vector difference, the decoding means decodesthe motion vector difference as the first side information, and wherein,when the indication information indicates that the first sideinformation does not contain the first motion vector difference, thefirst motion information reconstruction means generates the first motionvector predictor, based on the first motion vector predictor index, andreconstructs the generated first motion vector predictor as the firstmotion vector; and wherein, when the indication information indicatesthat the first side information contains the vector difference, thefirst motion information reconstruction means generates the first motionvector predictor, based on the first motion vector predictor index, andadds the generated first motion vector predictor to the decoded motionvector difference to generate and reconstruct the first motion vector.

The video predictive encoding device, video predictive encoding method,video predictive encoding program, video predictive decoding device,video predictive decoding method, and video predictive decoding programenable designation of one motion vector effective for the bi-prediction,based on the previously-encoded motion information, and thus achieve theeffect to enhance the performance of the bi-prediction with a smalleramount of coding bits.

Other systems, methods, features and advantages will be, or will become,apparent to one with skill in the art upon examination of the followingfigures and detailed description. It is intended that all suchadditional systems, methods, features and advantages be included withinthis description, be within the scope of the disclosure, and beprotected by the following claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of a video predictiveencoding device according to an embodiment.

FIG. 2 is a block diagram to illustrate an example of a predicted signalgeneration unit shown in FIG. 1.

FIG. 3 is a flowchart to illustrate an example of a first motionestimation process shown in FIG. 2.

FIG. 4 is a flowchart to illustrate an example of a zero-th motionestimation process shown in FIG. 2.

FIGS. 5A and 5B are schematic diagrams to illustrate examples ofencoding orders of frames.

FIGS. 6A through 6C are drawings to illustrate examples of referenceframe lists.

FIG. 7 is a drawing to illustrate an example of neighboring blocks.

FIG. 8 is a drawing to illustrate another example of neighboring blocks.

FIG. 9 is a flowchart showing an example of a video predictive encodingmethod of the video predictive encoding device shown in FIG. 1.

FIG. 10 is a block diagram showing an example of a video predictivedecoding device according to an embodiment.

FIG. 11 is a block diagram to illustrate an example of a motioninformation reconstruction unit shown in FIG. 10.

FIG. 12 is a flowchart to illustrate an example of a first motioninformation reconstruction process shown in FIG. 11.

FIG. 13 is a flowchart to illustrate an example of a zero-th motioninformation reconstruction process shown in FIG. 11.

FIG. 14 is a flowchart showing an example of a procedure of a videopredictive decoding method of the video predictive decoding device shownin FIG. 10.

FIGS. 15A and 15B are block diagram examples of modules used to executevideo predictive encoding according to an embodiment.

FIGS. 16A and 16B arc block diagram examples of modules used to executevideo predictive decoding method according to an embodiment.

FIG. 17 is a drawing showing an example hardware configuration of acomputer.

FIG. 18 is a perspective view of an example computer.

FIGS. 19A and 19B are schematic diagrams to illustrate an example of amotion estimation process in inter-frame prediction.

FIGS. 20A through 20I are schematic diagrams to illustrate an example ofintra-frame prediction.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the video predictive coding system will be describedbelow with reference to the accompanying drawings. In the description ofthe drawings identical or equivalent elements will be denoted by thesame reference signs, without redundant description. Furthermore,“frames,” “pictures,” and “images” (501 to 511 in FIG. 5) have the samemeaning in the description in the present specification.

FIG. 1 is a block diagram showing an example of a video predictiveencoding device 100 according to an embodiment of the predictive videocoding system. This video predictive encoding device 100 is providedwith circuitry that includes an input terminal 101, a block divisionunit 102, a predicted signal generation unit 103, a frame memory 104, asubtraction unit 105, a transform unit 106, a quantization unit 107, aninverse quantization unit 108, an inverse transform unit 109, anaddition unit 110, an encoding unit 111, an output terminal 112, and amotion information memory 113. The transform unit 106 and quantizationunit 107 can function as a residual signal compression unit or circuit,the inverse quantization unit 108 and inverse transform unit 109 canfunction as residual signal reconstruction unit or circuit, and themotion information memory can function as a motion information storageunit. The motion information memory 113 may be included in the predictedsignal generation unit 103. As used herein, the term “unit” isinterchangeable with the term “circuit” to describe hardware that mayalso execute software to perform the described functionality. The videopredictive encoding device 100 may be a computing device or computer,including circuitry in the form of hardware, or a combination ofhardware and software, capable of performing the describedfunctionality. The video predictive encoding device 100 may be one ormore separate systems or devices included in the video predictive codingsystem, or may be combined with other systems or devices within thevideo predictive coding system. In other examples, fewer or additionalunits may be used to illustrate the functionality of the predictivevideo encoding device 100. The input terminal 101 is a terminal thatimplements input of a signal of a video sequence consisting of aplurality of pictures.

The block division unit 102 divides a picture as an encoding target,which is represented by a signal input from the input terminal 101, intoa plurality of regions (target blocks or target regions). In the presentembodiment the encoding target picture is divided into blocks eachconsisting of 8×8 pixels, but the target picture may be divided intoblocks of any other size or shape. Furthermore, blocks of differentsizes may be mixed in a frame.

The predicted signal generation unit 103 detects motion information usedto generate a predicted signal of each predictive block in a targetblock and generates the predicted signal. Furthermore, it generates sideinformation used for reconstruction of the motion information in adecoding device. There are no restrictions on a predicted signalgeneration method, but methods applicable herein include the inter-frameprediction (uni-prediction or bi-prediction) and the intra-frameprediction (the intra-frame prediction is not illustrated) as describedin the background art.

In the present embodiment the predicted signal is generated bybi-prediction. A piece of motion information in the bi-prediction isdetected using a picture signal acquired via L104 so as to minimize thesum of absolute errors between the original signal of the target blockfed via L102 and the bi-predicted signal, by the block matching shown inFIG. 19. Then another piece of motion information is generated based onpreviously-encoded motion information.

Since the present embodiment describes the bi-prediction, the motioninformation is composed of zero-th motion information and first motioninformation, each of which contains a reference frame index (ref_idx[0]or ref_idx[1]) and a motion vector (mv[0][0/1] or mv[1][0/1]). Referenceframe candidates for the zero-th motion information are indicated byList0 in FIG. 6 and reference frame candidates for the first motioninformation are indicated by List1 in FIG. 6. [0/1] herein isinformation to identify a horizontal component and a vertical componentof each vector. The description of [0/1] will be omitted hereinafter(and also omitted similarly in the drawings).

The reproduced pictures to be registered in the reference frame listsshown in FIG. 6 may be automatically determined according to apredetermined rule or may be explicitly encoded in frame unit or insequence unit. On this occasion, the frame number can be used toidentify each reference frame as shown in FIG. 5 and FIG. 6.

The motion information generated by the predicted signal generation unit103 is output via L103 b to the motion information memory 113.

The motion information memory 113 stores the input motion information.The stored motion information is fed via L113 to the predicted signalgeneration unit to be used for encoding of motion information of asubsequent block.

The side information generated by the predicted signal generation unit103 is output via L103 c to the encoding unit 111.

The predicted signal generated by the predicted signal generation unit103 is output via L103 a to the subtraction unit 105 and to the additionunit 110.

The subtraction unit 105 subtracts the predicted signal for the targetblock fed via line L103 a, from the pixel signal of the target block fedvia line L102 after the division in the block division unit 102, togenerate a residual signal. The subtraction unit 105 outputs theresidual signal obtained by the subtraction, via line L105 to thetransform unit 106.

The transform unit 106 is a part that transforms the input residualsignal by a discrete cosine transform. The quantization unit 107 is apart that quantizes transform coefficients obtained by the discretecosine transform by the transform unit 106.

The encoding unit 111 entropy encodes the side information fed from thepredicted signal generation unit and the quantized transformcoefficients fed from the quantization unit 107, and the encoded data isoutput via L111 to the output terminal 112. There are no restrictions ona method of the entropy encoding, but applicable methods includearithmetic coding, variable-length coding, and so on.

The output terminal 112 outputs the information fed from the encodingunit 111, together to the outside.

The inverse quantization unit 108 inversely quantizes the quantizedtransform coefficients. The inverse transform unit 109 reconstructs aresidual signal by an inverse discrete cosine transform. The additionunit 110 adds the reconstructed residual signal to the predicted signalfed via L103 a, to reproduce a signal of the encoding target block, andstores the signal in the frame memory 104. The present embodimentemploys the transform unit 106 and the inverse transform unit 109, butit is also possible to use other transform processing in place of thesetransform units. It is also noted that the transform unit 106 and theinverse transform unit 109 are not always essential. In this manner, thereproduced signal of the encoding target block thus encoded isreconstructed by the inverse process and stored in the frame memory 104,in order to be used in generation of the predicted signal of thesubsequent encoding target block.

Next, the predicted signal generation unit 103 will be described indetail. Then, first, the motion information, predictive motioninformation, and side information will be described.

As described above, the motion information in the bi-prediction iscomposed of the zero-th motion information and the first motioninformation, each of which contains a reference frame index (ref_idx[0]or ref_idx[1]) and a motion vector (mv[0] or mv[1]). The reference framecandidates for the zero-th motion information are indicated by List0 inFIG. 6 and the reference frame candidates for the first motioninformation are indicated by List1 in FIG. 6.

In the bi-prediction of the present embodiment, the predicted signalgeneration unit 103 uses the previously-encoded motion information aspredictive motion information. The previously-encoded motion informationcontains motion information associated with neighboring blockspreviously encoded, and previously-encoded motion information of atarget region. The motion information associated with neighboring blocksrefers to motion information used in generation of the predicted signalwhen each neighboring block was an encoding target, and is stored in themotion information memory 113.

The predictive motion information is also composed of zero-th predictivemotion information and first predictive motion information, each ofwhich contains a reference frame index (ref_idx[0] or ref_idx[1]) and amotion vector (mv[0] or mv[1]). Reference frame candidates for thezero-th predictive motion information are indicated by List0 in FIG. 6and reference frame candidates for the first predictive motioninformation are indicated by List1 in FIG. 6.

A specific utilization method of the predictive motion information is togenerate a motion vector predictor, based on the zero-th predictivemotion information, in delta encoding of the motion vector of thezero-th motion information detected by block matching with reference tothe reference frames in List0. The first motion information using thereference frames in List1 is generated based on the first predictivemotion information.

An example of the predictive motion information will be described withreference to FIG. 7. A block 400 shown in FIG. 7 is a target block, andpieces of motion information associated with blocks 401 to 404 adjacentthereto are candidates for the predictive motion information. The motioninformation of each neighboring block contains the zero-th motioninformation and the first motion information. The both may be defined ascandidates for the predictive motion information or the predictivemotion information may be limited to either one of them (e.g., in thecase of prediction of N-th motion information, only the N-th motionvector of each neighboring block is defined as a candidate).

Furthermore, a block 410 represents a block located spatially at thesame position as the block 400, (or a co-located block) in a referenceframe. Pieces of motion information associated with the block 410 andblocks 411 to 415 adjacent thereto are candidates for the predictivemotion information. n represents a number to identify a candidate forthe predictive motion information and each selected candidate isindicated by a motion vector predictor index (mvp_idx[0] or mvp_idx[1]).In the present embodiment, the zero-th motion information is firstencoded and, for this reason, the zero-th motion information associatedwith the block 400 can also be used as the first predictive motioninformation (n=4 in the example).

The positions and numbers of the candidates for the predictive motioninformation can be those predefined between the encoder side and thedecoder side, and there are no restrictions thereon. The number ofcandidates for the predictive motion information may be predeterminedbetween the encoder side and the decoder side, or may be encoded andprovided to the decoder.

If a reference frame identified by ref_idx of the predictive motioninformation is different from a reference frame identified by ref_idx ofthe target block, a scaling process of the motion vector in thepredictive motion information may be performed based on the framenumbers of the encoding target frame and the two reference frames.Specifically, the motion vector in the predictive motion information isscaled in order to be converted into a motion vector designating thereference frame identified by ref_idx of the target block, and theconverted motion vector obtained by the conversion is used as a motionvector predictor (pmv[0][0/1] or pmv[1][0/1]). On this occasion, thereference frame index (ref_idx[0] or ref_idx[1]) in the predictivemotion information is updated to the reference frame index (ref_idx[0]or ref_idx[1]) of the target block. [0/1] herein is information toidentify a horizontal component and a vertical component of each vector.The description of [0/1] will be omitted hereinafter (and also omittedsimilarly in the drawings).

The side information is composed of zero-th side information and firstside information. The zero-th side information contains ref_idx[0], amotion vector difference (mvd[0][0/1]=mv[0][0/1]−pmv[0][0/1]), andmvp_idx[0]. The first side information contains ref_idx[1] andmvp_idx[1]. Since mv[1][0/1]=pmv[1][0/1] in the present embodiment,vector values of mvd[1][0/1] are always 0. Therefore, mvd[1][0/1] can bereconstructed on the decoder side without being encoded and, for thisreason, it does not have to be included in the first side information.[0/1] herein is information to identify a horizontal component and avertical component of each vector. The description of [0/1] will beomitted hereinafter (and also omitted similarly in the drawings).

FIG. 2 is a block diagram showing an example configuration of thepredicted signal generation unit 103 according to the presentembodiment. This predicted signal generation unit 103 is provided with afirst motion information estimation unit 121, a zero-th motioninformation estimation unit 122, and a predicted signal combining unit123.

The first motion information estimation unit 121 uses the referenceframes in List1 input via L104, to select a set of a first predictivemotion information candidate and a reference frame index to generate afirst predicted signal most similar to the original signal of the targetblock fed via L102, from candidates for the first predictive motioninformation fed via L113 (wherein motion vector predictors are obtainedby scaling motion vectors according to reference frame indexes). Thefirst motion information estimation unit 121 outputs the first predictedsignal via L121 a to the predicted signal combining unit 123 and outputsthe first motion information generated based on the selected firstpredictive motion information and reference frame index, via L121 b andvia L103 b 1 to the zero-th motion information estimation unit 122 andto the motion information memory 113, respectively. Furthermore, itgenerates first side information and outputs the first side informationvia L103 c 1 to the encoding unit 111.

The zero-th motion information estimation unit 122 uses the first motioninformation input via L121 b and the reference frames in List1 input viaL104, to generate the first predicted signal. Then the zero-th motioninformation estimation unit 122 searches the reference frames in List0input via L104, for a candidate for a zero-th predicted signal, anddetects zero-th motion information to minimize the sum of absolutedifferences between a bi-predicted signal generated from the firstpredicted signal and the candidate for the zero-th predicted signalobtained by the search, and the original signal of the target blockinput via L102. Then it outputs the zero-th predicted signal generatedfrom the detected zero-th motion information, via L122 to the predictedsignal combining unit 123. It also outputs the zero-th motioninformation via L103 b 2 to the motion information memory 113.Furthermore, it generates zero-th side information and outputs thezero-th side information via L103 c 2 to the encoding unit 111.

It is also allowable to first execute the processing by the zero-thmotion information estimation unit 122 to derive the zero-th motioninformation and the zero-th side information prior to the zero-thpredicted signal. In this case, the zero-th motion informationestimation unit 122 detects the zero-th motion information to minimizethe sum of absolute differences between a predicted signal generatedfrom the zero-th predicted signal candidate obtained by the search, andthe original signal of the target block input via L102. Then the firstmotion information estimation unit 121 executes the processing, usingthe zero-th predicted signal. For example, the first motion informationestimation unit 121 uses the reference frames in List1 input via L104,to generate a candidate for the first predicted signal from amongcandidates for the first predictive motion information input via L113(wherein motion vector predictors are obtained by scaling motion vectorsaccording to reference frame indexes), and selects a set of a firstpredictive motion information candidate and a reference frame index suchthat a bi-predicted signal generated from the zero-th predicted signaland the first predicted signal candidate most approximates, or issimilar to, the original signal of the target block input via L102. Thismodification can be implemented by feeding the zero-th motioninformation to the first motion information estimation unit 121.

The predicted signal combining unit 123 averages the first predictedsignal and the zero-th predicted signal input via L121 a and L122, togenerate a predicted signal of the target block and outputs thepredicted signal via L103 a to the subtraction unit 105 and the additionunit 110.

FIG. 3 shows a flowchart of example operation of the first motioninformation estimation unit 121. First, step S301 is to set M (M=4 inFIG. 6(A) and (C), or M=2 in FIG. 6(B)) for the number of referenceframes in List1 used in the prediction of the target block, and set 0for the reference frame index ref_idx[1] of List1 contained in the firstmotion information, to initialize a count m for the reference framenumber in List1 to 0. Furthermore, a motion vector evaluation value D isset at a Max value. Next, step S302 is to set N for the number ofcandidates for motion vector predictor (N=11 in FIG. 7, provided thatwhen the first motion information estimation unit 121 is carried outprior to the zero-th motion information estimation unit 122, n=4 isskipped because the zero-th motion information of block 400 is notdetermined yet), and set 0 for the motion vector predictor indexmvp_idx[1] contained in the first side information, to initialize acount n for the predictive motion information number to 0.

Next, step S303 is to derive a motion vector ofmotion vector predictorcandidate n from the motion vectors of the neighboring blocks and stepS304 is to generate the n-th motion vector predictor pmv[1][m][n][0/1](where [0/1] is information to identify a horizontal component and avertical component of the vector, and the description of [0/1] will beomitted hereinafter as well as in the drawings). On this occasion, themotion vector of the neighboring block is scaled according to a distancebetween the target frame and the reference frame (or according to framenumbers identified by reference frame indexes) to obtain the motionvector predictor. Thereafter, step S305 is to generate the predictedsignal of the target block, based on the m-th reference frame and then-th scaled motion vector predictor (pmv[1][m][n]), and step S306 is todetermine whether the sum of absolute differences of a residual signalbetween the generated predicted signal and the original signal of thetarget block is smaller than the motion vector evaluation value D. Whenthe sum of absolute differences is not less than the motion vectorevaluation value D, the processing proceeds to step S308. When the sumof absolute differences is smaller than the motion vector evaluationvalue D, the processing proceeds to step S307 to update the motionvector predictor index mvp_idx[1] contained in the first sideinformation, to n, update the reference frame index ref_idx[1] to m, andupdate the motion vector evaluation value D to the sum of absolutedifferences of the residual signal between the predicted signal and theoriginal signal of the target block calculated in step S306.Furthermore, the motion vector mv[1] in the first motion information isset to the motion vector predictor pmv[1][ref_idx[1]][mvp_idx[1]] andthe reference frame index is set to ref_idx[1]. Thereafter, theprocessing proceeds to step S308.

Step S308 is to determine whether the value of n is smaller than N; whenn is smaller than N, the processing proceeds to step S309; when nreaches N, the processing proceeds to step S310. Step S309 is to add 1to the value of n and then the processing returns to step S303.Thereafter, the steps from S303 to S307 are repeatedly carried out untiln reaches N. Step S310 is to determine whether the value of m is smallerthan M; when m is smaller than M, the processing proceeds to step S311to add 1 to the value of m and then returns to step S302. Thereafter,the steps from S302 to S309 are repeatedly carried out until m reachesM. When m reaches M, step S312 is carried out to output the first sideinformation (ref_idx[1], mvp_idx[1]) to the encoding unit 111, store thefirst motion information (ref_idx[1] and mv[1]) into the motioninformation memory 113, and output the first motion information to thezero-th motion information estimation unit 122, followed by end ofprocessing.

FIG. 4 shows a flowchart of example operation of the zero-th motioninformation estimation unit 122. First, step S351 is to generate thefirst predicted signal in the bi-prediction, based on the first motioninformation. Next, step S352 is to set M (M=4 in FIGS. 6(A) and (C), orM=2 in FIG. 6(B)) for the number of reference frames in List0 used inthe prediction of the target block, and set 0 for the reference frameindex ref_idx[0] of List0 contained in the zero-th motion information,to initialize the count m for the reference frame number in List0 to 0.Furthermore, the motion vector evaluation value D is set at a Max value.Next, step S353 is to determine the motion vector predictor indexmvp_idx[0] to identify a motion vector predictor used in differenceencoding of the zero-th motion vector, from a plurality of candidates. Aselection method herein can be, for example, the technique shown insteps S303 to S309 in FIG. 3. Then a motion vector predictor candidatepmv[0][m][n] is generated. On this occasion, a motion vector predictoris obtained by scaling the motion vector of the neighboring blockaccording to a distance between the target frame and the reference frame(or according to the frame numbers identified by reference frameindexes), as described in step S304 of FIG. 3.

Next, step S354 is to acquire the reference frame indicated byref_idx[0], which is stored in the frame memory 104, and to search forthe zero-th motion vector mv[0] to minimize the sum of absolutedifferences of the residual signal between the bi-predicted signalobtained by averaging together with the first predicted signal, and theoriginal signal. Subsequently, step S355 is to generate the zero-thmotion vector difference mvd[0] (=mv[0]−pmv[0][m][n]). Thereafter, stepS356 is to determine whether the total of the sum of absolutedifferences of the residual signal between the generated bi-predictedsignal and the original signal of the target block, and a code amountevaluation value of the zero-th side information (mvd[0] and m and n)(which is defined by λ(QP)×(an amount of coding bits of mvd, m, and n),where λ is a weight value determined by parameter QP to definequantization accuracy in quantization of transform coefficients obtainedby transform of a prediction error signal) is smaller than the motionvector evaluation value D. When the total of the sum of absolutedifferences +the code amount evaluation value is not less than themotion vector evaluation value D, the processing proceeds to step S358.When the total of the sum of absolute differences +the code amountevaluation value is smaller than the motion vector evaluation value D,the processing proceeds to step S357 to update the motion vectorpredictor index mvp_idx[0] in the zero-th side information to n, updatethe reference frame index ref_idx[0] to m, update the motion vectordifference mvd[0] to (mv[0]−pmv[0][ref_idx[1]][mvp_idx[1]]), and updateD to the total of the sum of absolute differences of the residual signalbetween the bi-predicted signal and the original signal of the targetblock+the code amount evaluation value calculated in step S356.Furthermore, the motion vector mv[0] in the zero-th motion informationis updated. Thereafter, the processing proceeds to step S358.

Step S358 is to determine whether the value of m is smaller than M; whenm is smaller than M, the processing proceeds to step S359 to add 1 tothe value of m, and returns to step S353. Thereafter, the steps fromS353 to S359 are repeatedly carried out until m reaches M. When mreaches M, step S360 is carried out to output the zero-th sideinformation (ref_idx[0], mvd[0], mvp_idx[0]) to the encoding unit 111and store the zero-th motion information (ref_idx[0] and mv[0]) into themotion information memory 113, followed by end of processing.

It is noted that the zero-th motion information estimation unit 122 mayfirst execute the processing to first determine the zero-th motioninformation and the zero-th side information prior to the zero-thpredicted signal. In this case, step S351 in FIG. 4 is omitted and stepS356 is modified to determine the sum of absolute differences of theresidual signal between the zero-th predicted signal, instead of thebi-predicted signal, and the original signal. In FIG. 3, it becomespossible to utilize the zero-th motion information indicated by n=4 inFIG. 7, as a candidate for the predictive motion information. Thismodification can be implemented by adding a step of generating thezero-th predicted signal in the bi-prediction based on the zero-thmotion information and modifying step S306 so as to calculate the sum ofabsolute differences of the residual signal between the bi-predictedsignal generated by averaging the first predicted signal and the zero-thpredicted signal thus generated, and the original predicted signal.

In this manner, the first motion vector in the bi-prediction isgenerated based on the previously-encoded motion information, and theidentification information to identify the reference frame index and thepredictive motion information from a plurality of candidates as shown inthe examples of FIGS. 6 and 7, is encoded instead of the motion vector;this method allows the encoding device to generate one similar signalwhich is similar to a signal of a target block of a texture region withhigh randomness or a flat region including noise, with a smaller amountof coding bits. Furthermore, the search is conducted on the referenceframe to detect and encode the zero-th motion vector to generate thesecond similar signal, whereby an effect to enhance the smoothing effectof bi-predicted signal can be expected, when compared to the case wherethe two motion vectors are obtained both from the previously-encodedmotion information.

FIG. 9 is a flowchart showing an example procedure of a video predictiveencoding method in the video predictive encoding device 100. First, theblock division unit 102 divides an input picture into 8×8 encodingblocks (the input picture may be divided into blocks of any other sizeor shape, or blocks of different sizes may be mixed in an inputpicture).

First, the first motion information estimation unit 121 forming thepredicted signal generation unit 103 generates the first predictedsignal similar to a target block, using the reference frames in List1obtained from the frame memory 104 and the candidates for firstpredictive motion information obtained from the motion informationmemory, and also generates the first motion information and the firstside information used for generation of the first predicted signal (stepS100). The details of this step were already described with FIG. 3.Next, the zero-th motion information estimation unit 122 generates thezero-th predicted signal similar to the target block, using thereference frames in List0 obtained from the frame memory 104 and thecandidates for zero-th predictive motion information obtained from themotion information memory, and the first motion information obtainedfrom the first motion information estimation unit 121, and alsogenerates the zero-th motion information and the zero-th sideinformation used for generation of the zero-th predicted signal (stepS150). The details of this step were already described with FIG. 4.

Next, the encoding unit 111 entropy encodes the zero-th side informationand the first side information and stores the zero-th motion informationand the first motion information into the motion information memory 113(step S101). Subsequently, in step S102, the predicted signal combiningunit 123 forming the predicted signal generation unit 103 averages thezero-th predicted signal and the first predicted signal to generate abi-predicted signal of the target block. A residual signal indicative ofa difference between the pixel signal of the encoding target block andthe predicted signal is transformed and encoded by the transform unit106, quantization unit 107, and encoding unit 111 (step S103). Theencoded data of the side information and quantized transformcoefficients is output via the output terminal 112 (step S104). Forpredictive encoding of a subsequent encoding target block, the inversequantization unit 108 and the inverse transform unit 109 decode theencoded residual signal after these processes, or in parallel with theseprocesses. Then the addition unit 110 adds the decoded residual signalto the predicted signal to reproduce a signal of the encoding targetblock. The reproduced signal is stored as a reference frame in the framememory 104 (step S105). Unless the processing is completed for allencoding target blocks, the processing returns to step S100 to performthe processing for the next encoding target block. When the processingis completed for all the encoding target blocks, the processing isterminated (step S106).

Next, a video predictive decoding method according to embodiments of thepredictive video coding system will be described. FIG. 10 is a blockdiagram showing an example of a video predictive decoding device 200according to the present embodiment. This video predictive decodingdevice 200 is provided with circuitry that includes an input terminal201, a decoding unit 202, an inverse quantization unit 203, an inversetransform unit 204, an addition unit 205, an output terminal 206, amotion compensation unit 207, a motion information reconstruction unit208, a frame memory 104, and a motion information memory 113. Theinverse quantization unit 203 and inverse transform unit 204 function asresidual signal reconstruction unit or circuit and the motioninformation memory 113 functions as motion information storage unit. Thedecoding function performed by the inverse quantization unit 203 and theinverse transform unit 204 may be any means other than these.Furthermore, the inverse transform unit 204 may be omitted. The videopredictive decoding device 200 may be a computing device or computer,including circuitry in the form of hardware, or a combination ofhardware and software, capable of performing the describedfunctionality. The video predictive decoding device 200 may be one ormore separate systems or devices included in the video predictive codingsystem, or may be combined with other systems or devices within thevideo predictive coding system. In other examples, fewer or additionalunits may be used to illustrate the functionality of the predictivevideo decoding device 200.

The input terminal 201 implements input of compressed data resultingfrom compression encoding by the aforementioned video predictiveencoding method. This compressed data contains encoded data ofinformation of quantized transform coefficients obtained bytransformation and quantization of error signals and entropy encoding oftransform coefficients, and encoded data of the zero-th side informationand the first side information for generation of hi-predicted signals ofblocks, for a plurality of divided encoding blocks.

The decoding unit 202 analyzes the compressed data input from the inputterminal 201, separates the data into the encoded data of quantizedtransform coefficients and the encoded data of side information abouteach decoding target block, performs entropy decoding thereof, andoutputs the decoded data via L202 a and via L202 b to the inversequantization unit 203 and to the motion information reconstruction unit208, respectively.

The motion information reconstruction unit 208 receives the zero-th sideinformation (ref_idx[0], mvd[0], mvp_idx[0]) and the first sideinformation (ref_idx[1], mvp_idx[1]) via L202 b and reconstructs thezero-th motion information (ref_idx[0], mv[0]) and the first motioninformation (ref_idx[1], mv[1]), using the previously-decoded motioninformation acquired via L113. The reconstructed zero-th motioninformation and first motion information is output via L208 a and viaL208 b to the motion compensation unit 207 and to the motion informationmemory 113, respectively. The motion information memory stores themotion information.

The motion compensation unit 207 acquires previously-reproduced signalsfrom the frame memory 104, based on the two pieces of motioninformation, and averages the two predicted signals to generate abi-predicted signal of the decoding target block. The predicted signalthus generated is output via L207 to the addition unit 205.

The quantized transform coefficients of the residual signal in thedecoding target block decoded by the decoding unit 202 are output viaL203 to the inverse quantization unit 203. The inverse quantization unit203 inversely quantizes the quantized coefficients of the residualsignal in the decoding target block. The inverse transform unit 204transforms the inversely quantized data by an inverse discrete cosinetransform to generate a residual signal.

The addition unit 205 adds the bi-predicted signal generated by themotion compensation unit 207, to the residual signal reconstructed bythe inverse quantization unit 203 and the inverse transform unit 204 andoutputs a reproduced pixel signal of the decoding target block via lineL205 to the output terminal 206 and the frame memory 104. The outputterminal 206 outputs the signal to the outside (e.g., a display).

The frame memory 104 stores the reproduced picture output from theaddition unit 205, as a reference frame, which is a reproduced picturefor reference for the next decoding process.

FIG. 11 is a block diagram showing an example configuration of themotion information reconstruction unit 208 according to the presentembodiment. This motion information reconstruction unit 208 is providedwith a first motion information reconstruction unit 211 and a zero-thmotion information reconstruction unit 212.

These first motion information reconstruction unit 211 and zero-thmotion information reconstruction unit 212 can operate simultaneously.

The zero-th motion information reconstruction unit 212 receives input ofthe zero-th side information (ref_idx[0], mvp_idx[0], mv[0]) to generatethe zero-th motion vector predictor (pmv[0][ref_idx[0]][mvp_idx[0]])from the motion information of the neighboring block obtained via L113,adds the motion vector predictor to the motion vector difference(mvd[0]) in the side information to generate the motion vector in thezcro-th motion information, thereby reconstructing the zero-th motioninformation. Similarly, when the reference frame identified by ref_idxis different from the reference frame identified by ref_idx of thetarget block, a scaling process of the motion vector in the predictivemotion information may be performed based on the frame numbers of theencoding target frame and the two reference frames.

The first motion information reconstruction unit 211 receives input ofthe first side information (ref_idx[1], mvp_idx[1]) to generate thefirst motion vector predictor (pmv[1][ref_idx[1]][mvp_idx[1]]) from thepreviously-decoded motion information obtained via L113. This motionvector predictor is defined as the motion vector in the first motioninformation (mv[1]=pmv[1][ref_idx[1]][mvp_idx[1]]), therebyreconstructing the first motion information. At this time, the firstmotion vector may be reconstructed by setting the motion vectordifference mvd[1] to a zero vector and adding it to the motion vectorpredictor. On this occasion, if the reference frame identified byref_idx is different from the reference frame identified by ref_idx ofthe target block, a scaling process of the motion vector in thepredictive motion information may be performed based on the framenumbers of the encoding target frame and the two reference frames. Forexample, the motion vector in the predictive motion information isscaled to be converted into a motion vector to the reference frameidentified by the target block, and the motion vector after theconversion is used as a motion vector predictor.

FIG. 13 shows a flowchart of example operation of the zero-th motioninformation reconstruction unit 212. First, step S451 is to inputdecoded data of the zero-th side information (ref_idx[0] and mvp_idx[0]and mvd[0]). Next, step S452 is to input the motion informationindicated by mvp_idx[0], from the motion information of the neighboringblocks as shown in the example of FIG. 7 (excluding the motioninformation of block 400 because it is undetermined), based onref_idx[0] and mvp_idx[0] thus input, and derive the motion vectorpredictor (pmv[0][ref_idx[0]][mvp_idx[0]]). At this time, if thereference frame identified by ref_idx is different from the referenceframe identified by ref_idx of the target block, a scaling process ofthe motion vector in the predictive motion information may be performedbased on the frame numbers of the encoding target frame and the tworeference frames. Thereafter, step S453 is to add the generated motionvector predictor to the motion vector difference to reconstruct thezero-th motion vector (mv[0]=pmv[0][ref_idx[0]][mvp_idx[0]] +mvd[0]).Finally, step S454 is to output the zero-th motion information(ref_idx[0] and mv[0]) to the motion compensation unit 207 and themotion information memory 113, followed by end of processing.

FIG. 12 shows a flowchart of example operation of the first motioninformation reconstruction unit 211. First, step S401 is to inputdecoded data of the first side information (ref_idx[1] and mvp_idx[1])and step S402 is to set 0 for the vector values of the motion vectordifference (mvd[1]). Next, step S403 is to input the motion informationindicated by mvp_idx[1] from the previously-decoded motion information(in which n=4 can be included) as shown in the example of FIG. 7, basedon ref_idx[1] and mvp_idx[1] thus input, and derive the motion vectorpredictor (pmv[1][ref_idx[1]][mvp_idx[1]]). At this time, if thereference frame identified by ref_idx is different from the referenceframe identified by ref_idx of the target block, a scaling process ofthe motion vector in the predictive motion information may be performedbased on the frame numbers of the encoding target frame and the tworeference frames. Thereafter, step S404 is to add the generated motionvector predictor to the motion vector difference to reconstruct thefirst motion vector (mv[1] pmv[1][ref_idx[1]][mvp_idx[1]+mvd[1]]).Finally, step S405 is to output the first motion information (ref_idx[1]and mv[1]) to the motion compensation unit 207 and the motioninformation memory 113, followed by end of processing. Since the vectorvalues of mvd[1] are 0 in this example, step S402 may be omitted andstep S404 may be modified so as to set the motion vector predictor tothe motion vector (mv[1]=pmv[1][ref_idx[1]][mvp_idx[1]]).

Next, the video predictive decoding method in the video predictivedecoding device 200 shown in FIG. 10 will be described using the exampleoperational flow diagram of FIG. 14. First, compressed data is input viathe input terminal 201 (step S201). Then the decoding unit 202 performsdata analysis of the compressed data, and performs entropy decodingthereof to decode the zero-th side information and the first sideinformation used for generation of the bi-predicted signals, and thequantized transform coefficients (step S202).

Next, the zero-th motion information reconstruction unit 212 forming themotion information reconstruction unit 208 reconstructs the zero-thmotion information, using the zero-th side information and the motioninformation of the neighboring blocks stored in the motion informationmemory 113 (step S250). The details of this step were already describedwith FIG. 13.

Subsequently, the first motion information reconstruction unit 211forming the motion information reconstruction unit 208 reconstructs thefirst motion information, using the first side information and themotion information of the neighboring blocks stored in the motioninformation memory 113 (step S200). The details of this step werealready described with FIG. 12.

Next, the motion compensation unit 207 generates the bi-predicted signalof the decoding target block, based on the reconstructed motioninformation, and stores the motion information into the motioninformation memory 113 (S207).

The inverse quantization unit 203 performs the inverse quantization ofthe quantized transform coefficients decoded by the decoding unit 202and the inverse transform unit 204 performs the inverse transform togenerate a reproduced residual signal (S208). Then the generatedbi-predicted signal is added to the reproduced residual signal togenerate a reproduced signal and this reproduced signal is stored forreproduction of the next decoding target block into the frame memory 104(step S209). The processes from S202 to S209 are repeatedly carried outas long as next compressed data exists (S210), and all the data isprocessed to the last.

Next, a technique of selectively using a plurality of bi-predictionmethods including the predictive video coding system will be described.The bi-prediction of encoding only one motion vector difference asdescribed above (which will be referred to as bi-prediction type 2) canbe used adaptively in combination with the conventional bi-prediction ofencoding two motion vector differences (which will be referred to asbi-prediction type 1) and the uni-prediction. These prediction systemscan be used with switching in frame unit, in slice unit of a group ofblocks, or in block unit. Switching processes applicable herein includea method of encoding switching information and a method of making thedecoder side determine the switching based on the frame numbers of thereference frames.

The switching process between bi-prediction type 1 and bi-predictiontype 2 can be implemented by adding to the first motion informationestimation unit 121 in FIG. 2, a block matching function and a functionto calculate a motion vector difference by subtracting a predictivevector from a motion vector detected by search, and by adding to thefirst motion information reconstruction unit in FIG. 11, a function toderive a motion vector difference from the decoding unit.

Specific methods will be described below.

(Switching in Frame/Slice Unit, and Encoding of Switching Information ofBi-prediction Type)

For switching between bi-prediction type 1 and bi-prediction type 2 inframe/slice unit, the switching information of bi-prediction type (e.g.,BiPred_type) is encoded as included in a header of a frame or a slice.

When all reference frame candidates are past frames in the display orderwith respect to the encoding target frame as in FIG. 5 (A),bi-prediction type 2 is effective and thus the encoder side selects useof bi-prediction type 2. Then the encoding device encodes indicationinformation to indicate bi-prediction type 2 (e.g., BiPred_type=1), inthe header information of a frame or a slice. On the other hand, whenthe reference frame candidates include a future frame in the displayorder with respect to the encoding target frame as in FIG. 5 (B), theencoding device selects bi-prediction type 1 and encodes indicationinformation to indicate bi-prediction type 1 (e.g., BiPred_type=0), inthe header information of a frame or a slice.

When bi-prediction type 1 is used, a motion vector mv[1] is searched forand mvd[1] is included in the first side information, similar to thezero-th side information (encoded data of ref_idx[0] and mvd[0] andmvp_idx[0]), as encoded together with ref_idx[1] and mvp_idx[1], inencoding of a block in a frame or in a slice. When bi-prediction type 2is used, ref_idx[0] and mvd[0] and mvp_idx[0] are encoded as the zero-thside information, and ref_idx[1] and mvp_idx[1] are encoded as the firstside information.

The decoder side switches the reconstruction method of mvd[1], indecoding of each block in a frame or in a slice, based on the indicationinformation decoded from the header information of the frame or theslice. For example, when the indication information indicative ofbi-prediction type 1 (e.g., BiPred_type=0) is decoded, the first sideinformation is decoded including mvd[1], together with ref_idx[1] andmvp_idx[1], in decoding of each block in the frame or in the slice. Whenthe indication information indicative of bi-prediction type 2 (e.g.,BiPred_type=1) is decoded, the first side information is decoded toobtain ref_idx[1] and mvp_idx[1] and 0 is set for horizontal andvertical vector values of mvd[1], in decoding of each block in the frameor in the slice. The zero-th side information is decoded to obtainref_idx[0] and mvd[0] and mvp_idx[0], regardless of the value of theindication information.

The switching method between bi-prediction type 1 and bi-prediction type2 on the encoder side does not have to depend on the method describedherein. For example, bi-prediction type 2 may be used if all thereference frames are future frames in the display order with respect tothe encoding target frame. It is also possible to adopt a method ofchecking encoding efficiencies in actual application of bi-predictiontype 1 and bi-prediction type 2 (e.g., evaluation values obtained byconverting the sum of squared differences of an encoding errorsignal+encoding bit count) and selecting a type with a higher encodingefficiency.

Furthermore, in the case of an encoding method of adaptively using aplurality of block sizes as shown in the example of FIG. 8, theindication information indicative of switching between bi-predictiontype 1 and bi-prediction type 2 may be individually sent for each ofdifferent block sizes, in the frame or slice header. For example, whenthe block sizes are 64×64, 32×32, 16×16, and 8×8, four pieces ofindication information are included.

When both of the uni-prediction and bi-prediction are applied to a blockprediction method, the encoder side selects a prediction type(uni-prediction or bi-prediction) and encodes it as included in the sideinformation. Then the switching process between bi-prediction type 1 andbi-prediction type 2 is carried out with only blocks for which thebi-prediction is selected. The decoder side performs the switchingprocess between bi-prediction type 1 and bi-prediction type 2 with onlyblocks for which the information indicative of bi-prediction as aprediction type is decoded.

(Switching in Frame/Slice Unit, and Determination Based on Frame Numbersof Reference Frames)

The reference frame candidates shown in the examples of FIG. 5(A) andFIG. 5(B) are the same on the encoder side and on the decoder side. Forthis reason, which of bi-prediction type 1 and bi-prediction type 2 isto be used can be determined based on the frame numbers of the referenceframe candidates and the frame number of the encoding target frame. Forexample, bi-prediction type 2 is used when all the reference framecandidates are past frames in the display order with respect to theencoding target frame; bi-prediction type 1 is used when the referenceframe candidates include a future frame in the display order withrespect to the encoding target frame. This method does not requiretransmission of indication information.

It is also possible to apply bi-prediction type 2 when all the referenceframe candidates are future frames in the display order with respect tothe encoding target frame.

(Switching in Block Unit, and Encoding of Switching Information)

In the case of switching between bi-prediction type 1 and bi-predictiontype 2 in block unit, the switching information of bi-prediction type(e.g., BiPred_block_type) is encoded as included in the side informationof each block.

Since bi-prediction type 2 is effective when two reference frames inbi-prediction are past frames in the display order with respect to theencoding target frame, the encoder side selects use of bi-predictiontype 2. Then the encoding device adds indication information indicativeof bi-prediction type 2 (e.g., BiPred_block_type=1) to the sideinformation of the block and encodes it together with ref_idx[0],mvd[0], and mvp_idx[0] of the zero-th side information and ref_idx[1]and mvp_idx[1] of the first side information. On the other hand, whenthe two reference frames in bi-prediction include a future frame in thedisplay order with respect to the encoding target frame, the encoderside selects bi-prediction type 1. Then the encoding device addsindication information indicative of bi-prediction type 1 (e.g.,BiPred_block_type=0) to the side information of the block and encodes ittogether with ref_idx[0], mvd[0], and mvp_idx[0] of the zero-th sideinformation and ref_idx[1], mvd[1] (mvd[1] is included in the first sideinformation), and mvp_idx[1] of the first side information.

The decoder side decodes the side information of the block including theswitching information of bi-prediction type (e.g., BiPred_block_type)and switches the reconstruction method of mvd[1], based on a decodedvalue. Specifically, when the indication information indicative ofbi-prediction type 1 (e.g., BiPred_block_type=0) is decoded as the sideinformation of the block, the decoding device decodes ref_idx[0] andmvd[0] and mvp_idx[0] as the zero-th side information and decodesref_idx[1] and mvd[1] and mvp_idx[1] as the first side information. Onthe other hand, when the indication information indicative ofbi-prediction type 2 (e.g., BiPred_block_type=1) is decoded as the sideinformation of the block, the decoding device decodes ref_idx[0] andmvd[0] and mvp_idx[0] as the zero-th side information, decodesref_idx[1] and mvp_idx[1] as the first side information, and set 0 forhorizontal and vertical vector values of mvd[1].

The switching method between bi-prediction type 1 and the bi-predictiontype 2 on the encoder side does not have to depend on the methoddescribed herein. For example, when both of the two reference frames arefuture frames in the display order with respect to the encoding targetframe, bi-prediction type 2 may be applied. It is also possible to adopta method of checking encoding efficiencies in actual application ofbi-prediction type 1 and bi-prediction type 2 (e.g., evaluation valuesobtained by converting the sum of squared differences of an encodingerror signal+encoding bit count) for each block, and selecting a typewith a higher encoding efficiency.

When both of the uni-prediction and bi-prediction are applied to theblock prediction method, the encoder side selects a prediction type(uni-prediction or bi-prediction) and encodes it as included in the sideinformation. Then the switching process between bi-prediction type 1 andbi-prediction type 2 is carried out with only blocks for which thebi-prediction is selected. The decoder side performs the switchingprocess between bi-prediction type 1 and bi-prediction type 2 with onlyblocks for which the information indicative of bi-prediction as aprediction type is decoded.

(Switching in Block Unit, and Determination Based on Frame Numbers ofReference Frames)

The reference frame candidates shown in the examples of FIG. 5(A) andFIG. 5(B) are the same on the encoder side and the decoder side. Forthis reason, which of bi-prediction type 1 and bi-prediction type 2 isto be used can be determined based on the frame numbers of the tworeference frames used in bi-prediction, which are encoded/decoded as theside information of block, and the frame number of the encoding targetframe. For example, bi-prediction type 2 is applied when the tworeference frames used in the bi-prediction both are past frames in thedisplay order with respect to the encoding target frame; bi-predictiontype 1 is applied when either or both of the two reference frames arefuture frames in the display order with respect to the encoding targetframe. This method does not require transmission of indicationinformation. Bi-prediction type 2 may be applied when both of the tworeference frames are future frames in the display order with respect tothe encoding target frame.

(Combination of Switching in Frame/Slice Unit and Switching in BlockUnit)

In frame/slice unit, indication information to indicate whether theswitching of bi-prediction type is to be performed in frame/slice unitor in block unit is encoded/decoded.

When the switching of bi-prediction type is carried out in frame/sliceunit, the switching information of bi-prediction type (e.g.,BiPred_type) is additionally encoded/decoded as included in a header ofa frame or a slice, as described above. On this occasion, in the case ofthe encoding method of adaptively using a plurality of block sizes asshown in the example of FIG. 8, the indication information to indicateswitching between bi-prediction type 1 and bi-prediction type 2 may beindividually sent for each of different block sizes, in the frame orslice header. For example, when the block sizes are 64×64, 32×32, 16×16,and 8×8, four pieces of indication information are encoded.

When the switching of bi-prediction type is carried out in block frameunit, the switching information of bi-prediction type (e.g.,BiPred_block_type) is additionally encoded/decoded as included in theside information of a block, in block unit, as described above. On thisoccasion, further, in the case of the encoding method of adaptivelyusing a plurality of block sizes as shown in the example of FIG. 8,indication information to indicate whether the switching process betweenbi-prediction type 1 and bi-prediction type 2 is to be carried out maybe individually transmitted for each of different block sizes, in theframe or slice header. For example, when the block sizes are 64×64,32×32, 16×16, and 8×8, four pieces of indication information areencoded.

It is also possible to encode/decode indication information to indicateapplication of only bi-prediction type 1, application of onlybi-prediction type 2, or switching between the two bi-prediction typesin each block in a frame/slice, in frame/slice unit. On this occasion,in the case of the encoding method of adaptively using a plurality ofblock sizes as shown in FIG. 8, the indication information may beindividually transmitted for each of different block sizes.

In another method, indication information to indicate whetherbi-prediction type 2 is to be applied is encoded/decoded in frame/sliceunit. In a frame/slice using bi-prediction type 2, indicationinformation may be further encoded/decoded to indicate whether switchingbetween bi-prediction type 1 and bi-prediction type 2 is to be performedin each block in the frame/slice, or to indicate whether bi-predictiontype 2 is applied to all the blocks in the frame/slice. On thisoccasion, in the case of the encoding method of adaptively using aplurality of block sizes as shown in FIG. 8, these pieces of indicationinformation may be individually transmitted for each of the differentblock sizes, in the frame or slice header.

In the above description, the zero-th motion information of the targetblock 400 was included in the candidates for the first motion vectorpredictor shown in FIG. 7. However, it is also possible to separatelyprepare as bi-prediction type 3 a method of defining the zero-th motioninformation as first predictive motion information, scaling thepredictive motion information, based on the first reference frame indexincluded in the zero-th motion information and the first reference frameindex included in the first motion information, and using the scaledresult as first motion information. For example, the scaling isperformed so that the motion vector included in the zero-th motioninformation becomes the motion vector of the reference frame indicatedby the reference frame index of the first motion information. In thiscase, the zero-th motion information of the target block does not haveto be included in the candidates for the first predictive motioninformation in bi-prediction type 2 (n=4 in FIG. 7 is excluded from thecandidates). When bi-prediction type 3 is applied, encoding/decoding ofthe first motion vector predictor index is not carried out. For example,the first side information is encoded/decoded, including the firstreference frame index (ref_idx[1]) but not including the motion vectordifference (mvd[1]) and the first motion vector predictor index(mvp_idx[1]).

An application method of bi-prediction type 3 may be to encode/decodeindication information to indicate switching among bi-prediction type 1,bi-prediction type 2, and bi-prediction type 3, in frame/slice unit orin block unit. It is also possible to encode/decode indicationinformation to indicate switching between bi-prediction type 1 andbi-prediction type 3, or, switching between bi-prediction type 2 andbi-prediction type 3, in frame/slice unit or in block unit.

It is also conceivable to employ a method of using it as a substitutefor bi-prediction type 1 in the above description. For example,bi-prediction type 3 is applied when the plurality of reference framecandidates include a future frame in the display order with respect tothe encoding target frame; bi-prediction type 2 is applied when theplurality of reference frame candidates all are past frames in thedisplay order with respect to the encoding target frame.

In another applicable method, when the reference frame indicated by thereference frame index in the zero-th motion information is differentfrom the reference frame indicated by the reference frame index in thefirst motion information, bi-prediction type 3 is applied instead ofbi-prediction type 2 (with no need for encoding/decoding of the firstmotion vector predictor index). On the other hand, when the referenceframe indicated by the reference frame index in the zero-th motioninformation is the same as the reference frame indicated by thereference frame index in the first motion information, bi-predictiontype 2 is applied (with encoding/decoding of the first motion vectorpredictor index).

The switching information of bi-prediction type does not have to belimited to the identification information such as BiPred_type orBiPred_block_type. It may be any information to indicate whetherencoding/decoding of motion vector difference is to be carried out. Forexample, flag information may be encoded/decoded to indicate whether mvdis included in the first side information or in the zero-th sideinformation like mvd_list1_zero_flag or mvd_list0_zero_flag. Whenbi-prediction type 1, bi-prediction type 2, and bi-prediction type 3 areswitched in frame/slice unit or in block unit, a flag to indicate thatmvd is not included in the side information is sent in application ofeither of bi-prediction type 2 and bi-prediction type 3. The switchingbetween bi-prediction type 2 and bi-prediction type 3 may be performedbased on the reference frame indexes as described above (bi-predictiontype 2 is applied when the reference frame indicated by the referenceframe index in the zero-th motion information is the same as thereference frame indicated by the reference frame index in the firstmotion information), and selection information may be furtherencoded/decoded.

FIG. 15 is a block diagram showing examples of modules that can be usedduring execution of circuitry for the video predictive encoding method.As shown in FIG. 15(A), the video predictive encoding program P100 isprovided with a block division module P101, a predicted signalgeneration module P103, a storage module P104, a subtraction moduleP105, a transform module P106, a quantization module P107, an inversequantization module P108, an inverse transform module P109, an additionmodule P110, an encoding module P111, and a motion information storagemodule P113. Furthermore, as shown in FIG. 15(B), the predicted signalgeneration module P103 is provided with a first motion informationestimation module P121, a zero-th motion information estimation moduleP122, and a predicted signal combining module P123. The functionsimplemented by execution of the above respective modules can be includedin the functions of the above-described video predictive encoding device100. For example, the functions provided by the respective modules inthe video predictive encoding program P100 can be included in thefunctions of the block division unit 102, predicted signal generationunit 103, frame memory 104, subtraction unit 105, transform unit 106,quantization unit 107, inverse quantization unit 108, inverse transformunit 109, addition unit 110, encoding unit 111, motion informationmemory 113, first motion information estimation unit 121, zero-th motioninformation estimation unit 122, and predicted signal combining unit123.

FIG. 16 is a block diagram showing examples of modules that can be usedduring execution of circuitry for the video predictive decoding method.As shown in FIG. 16(A), the video predictive decoding program P200 isprovided with a decoding module P201, a motion information decodingmodule P202, a motion compensation module P203, a motion informationstorage module P204, an invcrsc quantization module P205, an inversetransform module P206, an addition module P207, and a storage moduleP104. Furthermore, as shown in FIG. 16(B), the motion informationdecoding module P202 is provided with a first motion informationreconstruction module P211 and a zero-th motion informationreconstruction module P212.

The functions implemented by execution of the above respective modulescan be include in those of the components of the aforementioned videopredictive decoding device 200. For example, the functions provided bythe respective modules in the video predictive decoding program P200 canbe included in the functions of the decoding unit 202, motioninformation reconstruction unit 208, motion compensation unit 207,motion information memory 113, inverse quantization unit 203, inversetransform unit 204, addition unit 205, frame memory 104, first motioninformation reconstruction unit 211, and zero-th motion informationreconstruction unit 212.

The video predictive encoding program P100 or the video predictivedecoding program P200 configured as described above can be stored in abelow-described storage medium 10, working memory 14 and/or memory 16shown in FIGS. 17 and 18, and is executed by circuitry, such as includedin a computer described below.

FIG. 17 is a drawing showing an example of a hardware circuitryconfiguration of a computer for executing a program stored in a storagemedium and FIG. 18 an example of a perspective view of a computer forexecuting a program stored in a storage medium. The equipment forexecuting the program stored in the storage medium is not limited tocomputers, but may be a DVD player, a set-top box, a cell phone, or thelike provided with circuitry that includes a CPU and is configured toperform processing and control based on the circuitry or circuitry andsoftware.

As shown in FIG. 17, the computer 30 is provided with circuitry thatincludes a reading device 12 such as a flexible disk drive unit, aCD-ROM drive unit, or a DVD drive unit, a communication port such as auniversal serial bus port (USB), Bluetooth port, an infraredcommunication port, or any other type of communication port that allowscommunication with an external device, such as another computer ormemory device. The computer 30 may also include a working memory 14 thatmay include an operating system, a memory 16 that stores data, such asat least part of a program such as a program stored in the storingmedium 10. In addition, the working memory 14 and/or the memory 16 mayinclude the memory 104 and the memory 113. The working memory 14 andmemory 16 may be one or more computer readable storage medium that isother than a transitory signal, and can include a solid-state memorysuch as a memory card or other package that houses one or morenon-volatile memories, such as read-only memories. Further, the computerreadable medium can include a random access memory or other volatilere-writable memory. Additionally or alternatively, the computer-readablemedium can include a magneto-optical or optical medium, such as a diskor tapes or any other non-transitory information storage medium tocapture carrier wave signals such as a signal communicated over atransmission medium. A digital file attachment to an e-mail, stored in astorage medium, or other self-contained information archive or set ofarchives may be considered a non-transitory distribution medium that isa tangible computer readable storage medium. Accordingly, theembodiments are considered to include any one or more of acomputer-readable storage medium or a non-transitory distributionstorage medium and other equivalents and successor information storagemedia, in which data or instructions may be stored. In addition, thecomputer 30 may have user interface circuitry that includes, a monitorunit 18 such as a display, a mouse 20 and a keyboard 22 as inputdevices, a touch screen display, a microphone for receipt of voicecommands, a sensor, or any other mechanism or device that allows a userto interface with the computer 30. In addition, the circuitry of thecomputer 30 may include a communication device 24 fortransmission/reception of data and others, and a central processing unit(CPU) 26, or processor, to control execution of the program. Theprocessor 26 may be one or more one or more general processors, digitalsignal processors, application specific integrated circuits, fieldprogrammable gate arrays, digital circuits, analog circuits,combinations thereof, and/or other now known or later developedcircuitry and devices for analyzing and processing data. In an example,when the storing medium 10 is put into the reading device 12, thecomputer 30 becomes accessible to the video predictive encoding ordecoding program stored in the storing medium 10, through the readingdevice 12, and becomes able to operate as the video encoding device orthe video decoding device according to the present embodiment, based onthe image encoding or decoding program.

As shown in FIG. 18, the video predictive encoding program and the videodecoding program may be provided in the form of computer data signal 40superimposed on a carrier wave, through a network. In this case, thecomputer 30 stores the video predictive encoding program or the videodecoding program received through the communication device 24, into thememory 16 and becomes able to execute the video predictive encodingprogram or the video predictive decoding program.

In the predictive video coding system, it is possible to further employmodifications as described below.

(1) Relation Between Zero-th Side Information and First Side Information

In the above description the first side information contains ref_idx[1]and mvp_idx[1] and the zero-th side information contains ref_idx[0] andmvd[0] and mvp_idx[0]; however, this configuration may be reversed.Specifically, the first side information contains ref_idx[1] and mvd[1]and mvp_idx[1] and the zero-th side information contains ref_idx[0] andmvp_idx[0]. In this case, the zero-th predictive motion information isgenerated from the motion information of the neighboring blocks, thezero-th predicted signal is generated, and then a search is conductedfor the first motion information to minimize the evaluation value of thesum of absolute differences between the bi-predicted signal and theoriginal signal of the target block+the side information.

In another available method, the first side information containsref_idx[1] and mvp_idx[1] and the zero-th side information containsref_idx[0] and mvp_idx[0]. For example, the motion vector difference isnot encoded and the decoder side sets 0 for the horizontal and verticalvector values of the two motion vector differences. In another method,the zero-th motion vector predictor and the first motion vectorpredictor are set to the zero-th motion vector and the first motionvector, respectively.

(2) Reference Frames

In the above description, the reference frame index is encoded/decodedas included in the first side information, but it may be determinedbased on the reference frame index associated with the motioninformation of the neighboring block indicated by mvp_idx (for example,for the neighboring blocks 401 to 404 in FIG. 7, the associatedreference frame index in List1 is used as it is and, for the neighboringblocks 410 to 415, a scaling process is applied to a motion vector in aneighboring block, based on the difference between reference framesindicated by the reference frame indexes of the target block and theneighboring block). Furthermore, the reference frame index of the firstmotion information may be preliminarily determined. Since the referenceframe index can be uniquely reconstructed on the decoder side in thesecases, the reference frame index ref_idx does not have to be encoded asincluded in the first side information.

The selection of reference frames and the lists thereof are not limitedto the examples of FIG. 5 and FIG. 6. The reference frame lists may beencoded in frame/slice unit, and the number of reference frames does notalways have to be 4 as in FIG. 5, but may be carried out with any othernumber.

(3) Predictive Motion Information

The above embodiments used plural pieces ofpreviously-encoded/previously-decoded motion information as candidatesfor predictive motion information as shown in FIG. 7, but the number andpositions of neighboring blocks to be used are not limited to those.

The bi-prediction can also be carried out in a configuration wherein thezero-th motion vector of the neighboring block is included in thecandidates for the first motion vector predictor and the first motionvector of the neighboring block is included in the candidates for thezero-th motion vector predictor. The zero-th motion informationencoded/decoded before may be added to the candidates for firstpredictive motion information.

The predictive video coding system is not limited, either, as to whetherthe scaling process of motion vector is to be carried out when thereference frame index in the previously-encoded/previously-decodedmotion information is different from the reference frame index in themotion information associated with the target block.

When there is only one candidate for motion vector predictor, the motionvector predictor index mvp_idx does not have to be encoded as includedin the first side information.

In the predictive video coding system there are no restrictions on themethod of generating the motion vector predictor, either. For example,it is also possible to adopt as a motion vector predictor the median ofan odd number of candidates for motion vector predictor. Since themotion vector predictor is also uniquely determined on the decoder sidein this case, the motion vector predictor index mvp_idx does not have tobe encoded as included in the first side information.

The bi-prediction of the predictive video coding system can also becarried out in the case of the encoding/decoding method using aplurality of block sizes, as shown in FIG. 8, as long as a method fordetermining the candidates for predictive motion information is defined.For example, in the example of FIG. 8, it is possible to define blocks421 to 428 adjacent to the target block 400 as candidates for predictiveinformation, or to preliminarily determine a numbering method ofneighboring blocks and set motion information of neighboring blocks asmany as a number designated in frame unit or in slice unit, ascandidates for motion vector predictor.

(4) Zero-th Motion Information and Side Information

In the predictive video coding system there are no restrictions on theconfigurations of the zero-th motion information and the sideinformation; the reference frame index and the motion vector predictorindex may be set as fixed values, without being encoded, or they may bederived by a predetermined method.

As for the zero-th motion information and side information, similar tothe first motion information and side information, the decoder side mayset mv[0]=pmv[0] (or mvd[0]=0), without encoding mvd[0].

(5) Frame Numbers

In the above description the frame number (frame_num) is used toidentify each of the reference frames and encoding target frame, but anyother information may be used, without influence on implementation ofthe predictive video coding system, as long as it is information thatallows identification of each reference frame.

(6) Reconstruction of First Motion Vector Difference mvd[1]

In the above-described bi-prediction of the predictive video codingsystem, the vector values of the first motion vector difference mvd[1]are 0. For this reason, without encoding mvd[1], the decoder side sets 0for the vector values of mvd[1] or sets the motion vector mv[1] topmv[1]. Another effective embodiment can be a method of includingmvd[1]=0 in the first side information and efficiently entropy encodingzeros.

When arithmetic coding is used for the entropy encoding, for example,the zero-th motion vector difference and the first motion vectordifference are encoded/decoded by different probability models. Forexample, two probability models with different probability frequenciesof vector values 0 of motion vector difference are prepared forencoding/decoding of the motion vector difference. Then the secondprobability model with the higher frequency of vector values 0 of motionvector difference is used for encoding/decoding of the first motionvector difference, and the other first probability model is used forencoding/decoding of the zero-th motion vector difference. Furthermore,different probability models may also be prepared for horizontal andvertical vector values of motion vector difference.

In another applicable method, the second probability model is used foronly the first motion vector difference of each block to whichbi-prediction type 2 is applied, and the first probability model is usedfor encoding/decoding of the motion vector differences of the otherblocks.

When variable-length coding is applied, the zero-th motion vectordifference and the first motion vector difference are encoded/decodedusing different variable-length code tables. For example, twovariable-length code tables with different code lengths assigned tovector values 0 of the motion vector difference are prepared. Then thesecond variable-length code table with a shorter code length assigned tovector values 0 of the motion vector difference is used forencoding/decoding of the first motion vector difference, and the otherfirst variable-length code table is used for encoding/decoding of thezero-th motion vector difference. Furthermore, separate variable-lengthcode tables may be prepared for horizontal and vertical vector values ofmotion vector difference.

In another applicable method, the second variable-length code table isused for only the first motion vector difference of each block to whichbi-prediction type 2 is applied, and the first variable-length codetable is used for encoding/decoding of the motion vector differences ofthe other blocks.

(7) N-prediction

In the above description the prediction types of inter-frame predictionare uni-prediction and bi-prediction, but the predictive video codingsystem can also be applied to prediction methods of combining three ormore predicted signals. When the predicted signal is generated bycombining three or more predicted signals, the number of pieces of sideinformation without mvd may be any number not less than 1.

(8) Transform Unit and Inverse Transform Unit

The transform process of residual signal may be carried out in a fixedblock size or the transform process may be carried out in each ofsub-regions obtained by subdividing a target region into thesub-regions.

(9) Color Signal

There is no specific description about color format in the aboveembodiments, but the predicted signal generation process may also becarried out for color signal or color-residual signal, separately fromluminance signal. It may also be carried out in synchronization with theprocessing of luminance signal.

The predictive video coding system was described above in detail on thebasis of the embodiments thereof. It should be, however, noted that thepredictive video coding system is by no means limited to the aboveembodiments. The predictive video coding system can be modified invarious ways without departing from the scope and spirit of thedescribed embodiments.

LIST OF REFERENCE SIGNS

100: video predictive encoding device; 101: input terminal; 102: blockdivision unit; 103: predicted signal generation unit; 104: frame memory;105: subtraction unit; 106: transform unit; 107: quantization unit; 108:inverse quantization unit; 109: inverse transform unit; 110: additionunit; 111: encoding unit; 112: output terminal; 113: motion informationmemory; 121: first motion information estimation unit; 122: zero-thmotion information estimation unit; 123: predicted signal combiningunit; 201: input terminal; 202: decoding unit; 203: inverse quantizationunit; 204: inverse transform unit; 205: addition unit; 206: outputterminal; 207: motion compensation unit; 208: motion informationreconstruction unit; 211: first motion information reconstruction unit;212: zero-th motion predictive information reconstruction unit.

What is claimed is:
 1. A video predictive decoding device comprising: adecoding unit which decodes side information and a compressed data of aresidual signal of a target region, which is a target to be decoded, outof plural sets of compressed data obtained by encoding a plurality ofdivided regions; a motion information reconstruction unit whichreconstructs a motion vector used to generate a predicted signal of thetarget region from the side information; a motion information storingunit which stores the motion vector; a motion compensation unit whichgenerates the predicted signal of the target region, based on the motionvector; a residual signal reconstruction unit which reconstructs areproduced residual signal of the target region from the compressed datacomprising the residual signal; and a picture storing unit which addsthe predicted signal to the reproduced residual signal to reconstruct apixel signal of the target region, and which stores the reconstructedpixel signal as a previously-reproduced picture, wherein the decodingunit decodes zero-th side information and first side information,wherein the zero-th side information contains a zero-th motion vectordifference, and a zero-th motion vector predictor index used to identifyas a zero-th motion vector predictor one motion vector selected from aplurality of motion vectors stored in the motion information storingunit, wherein the first side information contains a first motion vectorpredictor index used to identify as a first motion vector predictor onemotion vector selected from a plurality of motion vectors stored in themotion information storing unit, wherein the motion informationreconstruction unit comprises: a zero-th motion informationreconstruction unit, which derives the zero-th motion vector predictorbased on the zero-th motion vector predictor index by using the onemotion vector selected from the motion vectors stored in the motioninformation storing unit, and which adds the zero-th motion vectorpredictor to the zero-th motion vector difference to reconstruct azero-th motion vector; and a first motion information reconstructionunit, which sets 0 for a vector value of a first motion vectordifference, derives the first motion vector predictor based on the firstmotion vector predictor index by using the one motion vector selectedfrom the motion vectors stored in the motion information storing unit,and adds the first motion vector predictor to the first motion vectordifference to reconstruct a first motion vector, and wherein the motioncompensation unit combines two signals acquired from thepreviously-reproduced picture, based on the zero-th motion vector andthe first motion vector respectively, to generate the predicted signalof the target region.
 2. A video predictive decoding method executed bya video predictive decoding device, comprising: a decoding step ofdecoding side information and a compressed data of a residual signal ofa target region, which is a target to be decoded, out of plural sets ofcompressed data obtained by encoding a plurality of divided regions; amotion information reconstruction step of restoring a motion vector usedto generate a predicted signal of the target region from the sideinformation; a motion information storing step of storing the motionvector in a motion information storing unit; a motion compensation stepof generating the predicted signal of the target region, based on themotion vector; a residual signal reconstruction step of restoring areproduced residual signal of the target region from the compressed datacomprising the residual signal; and a picture storing step of adding thepredicted signal to the reproduced residual signal to reconstruct apixel signal of the target region, and storing the reconstructed pixelsignal as a previously-reproduced picture, wherein in the decoding step,the video predictive decoding device decodes zero-th side informationand first side information, wherein the zero-th side informationcontains a zero-th motion vector difference, and a zero-th motion vectorpredictor index used to identify as a zero-th motion vector predictorone motion vector selected from a plurality of motion vectors stored inthe motion information storing unit, wherein the first side informationcontains a first motion vector predictor index used to identify as afirst motion vector predictor one motion vector selected from aplurality of motion vectors stored in the motion information storingunit, wherein the motion information reconstruction step comprises: azero-th motion information reconstruction step of deriving the zero-thmotion vector predictor based on the zero-th motion vector predictorindex by using the one motion vector selected from the motion vectorsstored in the motion information storing unit, and adding the zero-thmotion vector predictor to the zero-th motion vector difference toreconstruct a zero-th motion vector; and a first motion informationreconstruction step of setting 0 for a vector value of a first motionvector difference, deriving the first motion vector predictor based onthe first motion vector predictor index by using the one motion vectorselected from the motion vectors stored in the motion informationstoring unit, and adding the first motion vector predictor to the firstmotion vector difference to reconstruct a first motion vector, andwherein in the motion compensation step, the video predictive decodingdevice combines two signals acquired from the previously-reproducedpicture, based on the zero-th motion vector and the first motion vectorrespectively, to generate the predicted signal of the target region.